Methods of determining status of position in shelf, shelves and non-transitory computer-readable storage media

ABSTRACT

The present disclosure provides a method of determining a status of a position on a shelf, a shelf and a non-transitory computer-readable storage medium. The method includes: acquiring current detection data of the shelf in a current detection period; determining first status information of respective positions from the current detection data, in response to determining that the current detection data satisfies a preset condition; in response to the current detection data indicating that a target position whose first status information indicates the second status exists, acquiring first status information of the target position from previous detection data in a previous detection period; and determining second status information of the target position based on the first status information of the target position in the current detection data and the previous detection data respectively.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims a priority of the Chinese patentapplication No. 202010590594.1 filed on Jun. 24, 2020 and entitled“METHODS AND APPARATUSES FOR DETERMINING STATUS OF POSITION IN SHELF,AND SHELVES”, which is incorporated herein by reference in its entirety.

TECHNICAL FIELD

The present disclosure relates to the field of data processingtechnology, and in particular to a method of determining a status of aposition on a shelf, a shelf and a non-transitory computer-readablestorage medium.

BACKGROUND

At present, more and more new types of shelves have been used inscenarios such as new retail, exhibitions, and precious metal displaysin a bank. By placing objects in a plurality of positions on the shelf,it may be convenient for users to pick and place the objects bythemselves. In order to facilitate the users to learn about an inventoryof the objects on the shelf, the shelf may show a position on the shelfas vacant when an object leaves the position, thereby ensuring aninteractive effect.

SUMMARY

The present disclosure provides a method of determining a status of aposition on a shelf, a shelf and a non-transitory computer-readablestorage medium.

In a first aspect, embodiments of the present disclosure provide amethod of determining a status of a position on a shelf, the positionbeing operatively configured to accommodate and display an object, themethod comprising:

acquiring current detection data of the shelf in a current detectionperiod, wherein the current detection data comprises data of one or morepositions on the shelf;

determining first status information of respective positions from thecurrent detection data, in response to determining that the currentdetection data satisfies a preset condition, wherein the first statusinformation indicates a first status that there is an object in aposition or a second status that there is no object in a position;

in response to the current detection data indicating that a targetposition whose first status information indicates the second statusexists, acquiring first status information of the target position fromprevious detection data in a previous detection period; and

determining second status information of the target position based onthe first status information of the target position in the currentdetection data and the previous detection data respectively, wherein thesecond status information indicates a third status that an object is outof stock or a fourth status that an object is picked up.

Optionally, determining that the current detection data satisfies thepreset condition, comprises:

determining whether the current detection data comprises motion data;

acquiring a first time interval between time of the current detectionperiod and time of a last detection period containing the motion data,in response to determining that the current detection data does notcomprise the motion data; and

determining that the current detection data satisfies the presetcondition when the first time interval exceeds a first preset timeinterval.

Optionally, the current detection data comprises a current frame imagecollected by a camera; and

determining whether the current detection data comprises the motiondata, comprises:

acquiring a difference in grayscale values between the current frameimage and a previous frame image at a same pixel;

determining a number of pixels with the difference in grayscale valuesexceeding a preset grayscale threshold; and

determining that the current frame image comprises the motion datarepresenting a motion relative to the previous frame image, in responseto a ratio of the number to a total number of pixels in the currentframe image exceeding a preset ratio threshold, otherwise determiningthat the current frame image does not comprise the motion data.

Optionally, determining the second status information of the targetposition based on the first status information of the target position inthe current detection data and the previous detection data respectively,comprises:

acquiring a second time interval between time of the current detectionperiod and time of a last detection period containing motion data, inresponse to the first status information of the target position in theprevious detection data indicating the second status; and

determining that the second status information of the target positionindicates the third status in response to the second time intervalexceeding a second preset time interval.

Optionally, determining the second status information of the targetposition based on the first status information of the target position inthe current detection data and the previous detection data respectively,comprises:

determining that the second status information of the target positionindicates the fourth status, in response to the first status informationof the target position in the previous detection data indicating thefirst status.

In a second aspect, the embodiments of the present disclosure provide ashelf comprising:

a main body provided with a plurality of layers, each of which isprovided with a plurality of positions that are operatively configuredto accommodate and display objects;

a data collector configured to collect current detection data of theshelf in a current detection period, wherein the current detection datacomprises data of one or more positions on the shelf; and

a processor coupled with the data collector and configured to:

acquire the current detection data;

determine first status information of respective positions from thecurrent detection data, in response to determining that the currentdetection data satisfies a preset condition, wherein the first statusinformation indicates a first status that there is an object in aposition or a second status that there is no object in a position;

in response to the current detection data indicating that a targetposition whose first status information indicates the second statusexists, acquire first status information of the target position fromprevious detection data in a previous detection period; and

determine second status information of the target position based on thefirst status information of the target position in the current detectiondata and the previous detection data respectively, wherein the secondstatus information indicates a third status that an object is out ofstock or a fourth status that an object is picked up.

Optionally, when determining that the current detection data satisfiesthe preset condition, the processor is further configured to:

determine whether the current detection data comprises motion data;

acquire a first time interval between time of the current detectionperiod and time of a last detection period containing the motion data,in response to determining that the current detection data does notcomprise the motion data; and

determine that the current detection data satisfies the preset conditionwhen the first time interval exceeds a first preset time interval.

Optionally, the data collector comprises a camera, and the currentdetection data comprises a current frame image collected by the camera;and

when determining whether the current detection data comprises the motiondata, the processor is further configured to:

acquire a difference in grayscale values between the current frame imageand a previous frame image at a same pixel;

determine a number of pixels with the difference in grayscale valuesexceeding a preset grayscale threshold; and

determine that the current frame image comprises the motion datarepresenting a motion relative to the previous frame image, in responseto a ratio of the number to a total number of pixels in the currentframe image exceeding a preset ratio threshold, otherwise determiningthat the current frame image does not comprise the motion data.

Optionally, when determining the second status information of the targetposition based on the first status information of the target position inthe current detection data and the previous detection data respectively,the processor is further configured to:

acquire a second time interval between time of the current detectionperiod and time of a last detection period containing motion data, inresponse to the first status information of the target position in theprevious detection data indicating the second status; and

determine that the second status information of the target positionindicates the third status in response to the second time intervalexceeding a second preset time interval.

Optionally, when determining the second status information of the targetposition based on the first status information of the target position inthe current detection data and the previous detection data respectively,the processor is further configured to:

determine that the second status information of the target positionindicates the fourth status in response to the first status informationof the target position in the previous detection data indicating thefirst status.

Optionally, the data collector comprises a plurality of pressure sensorsprovided at the bottom of the respective positions, and the currentdetection data comprises current pressure data collected by theplurality of pressure sensors.

Optionally, the shelf further comprises a display provided at a presetlocation of the main body and configured to display the first statusinformation or the second status information of the respectivepositions.

In a third aspect, the embodiments of the present disclosure provide ashelf comprising:

a main body provided with a plurality of layers, each of which isprovided with a plurality of positions that are operatively configuredto accommodate and display objects;

a camera provided on the top of the main body, having a field of viewcovering all of the positions on the main body, and configured tocapture current frame images of one or more positions; and

a processor coupled with the camera and configured to acquire firststatus information or second status information of respective positionsbased on the current frame images, or

the shelf comprising:

the main body provided with the plurality of layers, each of which isprovided with the plurality of positions that are operatively configuredto accommodate and display objects;

a plurality of pressure sensors provided at the bottom of the respectivepositions, and configured to acquire current pressure data of therespective positions; and

a processor coupled with the plurality of pressure sensors andconfigured to acquire the first status information or the second statusinformation of the respective positions based on the current pressuredata.

Optionally, the shelf further comprises a display provided at a presetlocation of the main body and configured to display the first statusinformation or the second status information of the respectivepositions.

In a fourth aspect, the embodiments of the present disclosure provide anon-transitory computer-readable storage medium having a computerexecutable program stored therein, wherein the computer executableprogram, when executed by a processor, causes the processor to performthe above method.

It should be understood that the above general description and thefollowing detailed description are merely exemplary and explanatory, andare not intended to limit the present disclosure.

BRIEF DESCRIPTION OF DRAWINGS

The drawings herein, which are incorporated into the specification andconstitute a part of the specification, illustrate embodiments inaccordance with the present disclosure, and are used together with thespecification to explain the principle of the present disclosure.

FIG. 1 is a schematic diagram illustrating a shelf according to anembodiment of the present disclosure.

FIG. 2 is a schematic diagram illustrating a mounting position of acamera according to an embodiment of the present disclosure.

FIG. 3 is a schematic diagram illustrating a mounting position of acamera according to another embodiment of the present disclosure.

FIG. 4 is a schematic diagram illustrating a shelf according to anotherembodiment of the present disclosure.

FIG. 5 is a flowchart illustrating a method of determining a status of aposition on a shelf according to an embodiment of the presentdisclosure.

FIG. 6 is a flowchart of determining whether current detection datasatisfies a preset condition according to an embodiment of the presentdisclosure.

FIG. 7 is a flowchart of determining whether current detection dataincludes motion data according to an embodiment of the presentdisclosure.

FIG. 8 is a flowchart of determining second status information of atarget position according to an embodiment of the present disclosure.

FIG. 9 is a schematic diagram illustrating an effect of displayingstatus information of a position according to an embodiment of thepresent disclosure.

FIG. 10 is a diagram illustrating an application scenario according toan embodiment of the present disclosure.

FIG. 11 is a block diagram illustrating an apparatus for determining astatus of a position on a shelf according to an embodiment of thepresent disclosure.

DETAILED DESCRIPTION

Exemplary embodiments will be described in detail herein, and examplesthereof are illustrated in the drawings. When the following descriptionrefers to the drawings, the same numbers in different drawings indicatethe same or similar elements, unless otherwise indicated. Theimplementations described in the following exemplary embodiments do notrepresent all implementations in accordance with the present disclosure.Rather, they are merely examples of apparatuses and methods inaccordance with some aspects of the present disclosure as detailed inthe appended claims.

When an object leaves a position on a shelf, the shelf may show theposition as vacant. However, when the object leaves the position, theobject may be picked up (for example, when a user purchases the object),or the object may be out of stock, which may not accurately reflect astatus of the position and thus affect an interactive experience.

In view of this, an embodiment of the present disclosure provides ashelf. Referring to FIG. 1 , the shelf 1 includes a main body 10, acamera 20 and a processor 30.

The main body 10 is provided with a plurality of layers, each of whichis provided with a plurality of positions that are operativelyconfigured to accommodate and display objects, such as positions 11, 12,and 13, positions 21, 22, and 23, . . . , positions 51, 52, and 53 inFIG. 1 . The first number of a reference numeral indicating a positionrepresents which layer the position is located on, and the second numberrepresents an ordinal number of the position on this layer. For example,the position 11 indicates the first position on the first layer.

The camera 20 is provided on the top of the main body 10, has a field ofview (space enclosed by four dashed lines in FIG. 1 ) covering all ofthe positions on the main body 10, and is configured to capture currentframe images of one or more positions.

The processor 30 is coupled with the camera 20, and configured toacquire first status information or second status information ofrespective positions based on the current frame images. The first statusinformation indicates a first status that there is an object in aposition or a second status that there is no object in a position. Thesecond status information indicates a third status that an object is outof stock or a fourth status that an object is picked up.

In an embodiment, the shelf 1 may further include a display 40 with anLCD display screen, an LED display screen, or an OLED display screen.The display 40 may be provided at a preset location of the main body 10so that a user in front of the shelf 1 may see contents displayedthereon, and the display 40 may be configured to display the firststatus information or the second status information of the respectivepositions. For example, a center point of the preset location may belevel with the user's line of sight when the user is standing.Considering that a height of the user is approximately between 120 cmand 200 cm, a height of the preset location may be set between 130 cmand 170 cm.

It should be noted that the camera 20 and the processor 30 are denotedwith dashed boxes, indicating that they are provided behind the display40, and the user cannot directly see the camera 20 and the processor 30when watching the contents displayed on the display 40. In animplementation, the camera 20 may also be provided on a bottom layer ofthe shelf 1 with its field of view covering the respective positions.

It should be noted that, in an implementation, the camera 20 mayautomatically adjust a focal length thereof so that the field of viewmay cover one or more positions to be captured. In this way, the currentdetection data may include data of the one or more positions.Alternatively, the camera 20 may capture an image involving all of thepositions on the shelf 1, and the processor 30 may divide the image toobtain an image involving the one or more positions. Those skilled inthe art may select an appropriate manner according to a specificscenario, and any scheme, in which the current detection data of the oneor more positions may be obtained, falls within the protection scope ofthe present disclosure. Alternatively, one camera 20 may be provided oneach layer of the main body 10, as shown in FIGS. 2 and 3 . The cameras20 in FIG. 2 are each provided on the top of a respective layer, and thecameras 20 in FIG. 3 are each provided on the bottom of a respectivelayer.

An embodiment of the present disclosure further provides another shelf.Referring to FIG. 4 , the shelf 1′ includes a main body 10, a pluralityof pressure sensors 20′, and a processor 30. The shelf 1′ differs fromthe shelf 1 shown in FIG. 1 in that: the pressure sensors 20′ areprovided in respective positions and configured to acquire currentpressure data of the respective positions, and the processor 30 iscoupled with the pressure sensors 20′ and configured to acquire thefirst status information or the second status information of therespective positions based on the current pressure data. When there isan object placed in a position, the pressure sensor 20′ may detectpressure data exceeding a pressure threshold; and when there is noobject placed in a position, the pressure sensor 20′ may detect pressuredata less than the pressure threshold. For other same components,reference may be made to the embodiment shown in FIG. 1 , which will notbe repeated herein.

Based on the shelves shown in FIGS. 1 and 4 , an embodiment of thepresent disclosure further provides a method of determining a status ofa position on a shelf, the position being operatively configured toaccommodate and display an object. FIG. 5 is a flowchart illustrating amethod of determining a status of a position on a shelf according to anembodiment of the present disclosure. Referring to FIG. 5 , the methodof determining a status of a position on a shelf includes steps 51-54.

At step 51, current detection data of the shelf in a current detectionperiod is acquired, where the current detection data includes data ofone or more positions on the shelf.

In this example, the processor 30 may acquire the current detection dataof the shelf in the current detection period in real time orperiodically. The processor 30 may be implemented by a server or anembedded system, which is not limited herein.

In an implementation, a device configured to collect the currentdetection data may be a data collector such as a camera or a pressuresensor. For example, in the shelf shown in FIG. 1 , the camera 20 maycapture current frame images of one or more positions in real time orperiodically, and in this case, the current detection data may includethe current frame images. For another example, in the shelf shown inFIG. 2 , the pressure sensors 20′ may acquire pressure data of one ormore positions in real time or periodically to obtain current pressuredata in the current detection period, and in this case, the currentdetection data may include the current pressure data.

At step 52, first status information of respective positions isdetermined from the current detection data, in response to determiningthat the current detection data satisfies a preset condition, where thefirst status information indicates a first status that there is anobject in a position or a second status that there is no object in aposition.

In this example, the processor 30 may determine whether the currentdetection data acquired at step 51 satisfies the preset condition, whichmay include steps 61-63 as shown in FIG. 6 .

At step 61, the processor 30 may determine whether the current detectiondata includes motion data. When a change in data of the same position inthe current detection data and previous detection data in a previousdetection period exceeds a threshold, it may be determined that the dataof the position in the current detection data is the motion data.

In this example, the current detection data including the current frameimages is taken as an example to determine whether the current detectiondata includes the motion data, which may include steps 71-73 as shown inFIG. 7 .

At step 71, the processor 30 may acquire a difference in grayscalevalues between the current frame image and a previous frame image at thesame pixel. At step 72, the processor 30 may determine the number ofpixels with the difference in grayscale values exceeding a presetgrayscale threshold. In an implementation, the preset grayscalethreshold may be set to 50. At step 73, in response to a ratio of thenumber to a total number of pixels in the current frame image exceedinga preset ratio threshold, the processor 30 may determine that thecurrent frame image includes the motion data representing a motionrelative to the previous frame image, otherwise, the processor 30 maydetermine that the current frame image does not include the motion data.In an implementation, the preset ratio threshold may be set to 30%.

It should be noted that, the processor 30 may perform motion detectionon the entire frame image at steps 71-73. However, the processor 30 mayacquire sub-images of the respective positions from the current frameimage, and then perform steps 71-73 on each sub-image. Since thesub-image from which a background area has been removed has a relativelysmall area, it may be processed at a relatively high speed. Thoseskilled in the art may select appropriate schemes according to specificscenarios, which are not limited herein.

At step 62, the processor 30 may acquire a first time interval betweentime of the current detection period and time of the last detectionperiod containing the motion data, in response to determining that thecurrent detection data does not include the motion data. The lastdetection period refers to a detection period before the currentdetection period. Since detection data acquired in each detection periodmay include motion data or may not include motion data, the lastdetection period containing the motion data refers to a detection periodthat detection data contains the motion data and having the shortesttime interval from detection time of the current detection period. Thatis, none of detection data acquired between the current detection periodand the last detection period containing the motion data contains themotion data.

At step 63, the processor 30 may determine that the current detectiondata satisfies the preset condition when the first time interval exceedsa first preset time interval.

The embodiment shown in FIG. 6 indicates that, the preset conditionmeans that neither the current detection data nor detection data withinthe first preset time interval before the current detection periodcontains the motion data, in the case that the current detection dataincludes the current frame image.

It may be understood that when the current detection data includespressure data collected by the pressure sensors in the shelf, motiondetection may not be performed on the current detection data,considering that picking up an object is an instantaneous action. Inother words, the preset condition means that no motion detection isperformed in the case that the current detection data includes thecurrent pressure data.

In this example, a status discriminator may be pre-stored in the shelf1. The status discriminator may be implemented by a neural networkmodel, a template difference model or a template matching model, whichmay be set according to specific scenarios. The processor 30 may inputthe current detection data to the status discriminator, and acquirefirst status information of respective positions output from the statusdiscriminator. For example, the first status information may indicate afirst status that there is an object in a position. For another example,the first status information may indicate a second status that there isno object in a position.

Taking the template difference model as an example, the statusdiscriminator may include a standard template that contains detectiondata collected when no object is placed in any position. Taking thedetection data including an image as an example, the respectivepositions in the standard template are vacant, and the statusdiscriminator may obtain a difference image by subtracting the standardtemplate from the current frame image. If a grayscale difference of eachpixel for a position in the difference image is relatively large (forexample, more than a preset threshold such as 10), it means that thereis an object placed in this position. If a grayscale difference of eachpixel for a position in the difference image is relatively small (forexample, less than the preset threshold such as 10), it means that thereis no object placed in this position, and this position is vacant.Setting the preset threshold as 10 is to match a deviation betweenvarious images captured under different light. Taking the detection dataincluding pressure data as an example, the respective positions in thestandard template are vacant, and the status discriminator may obtain adifference pressure by subtracting the standard template from thecurrent pressure data. If the difference pressure in a position isrelatively large (for example, more than a preset threshold such as 10gram), it means that there is an object placed in this position. If thedifference pressure in a position is relatively small (for example, lessthan the preset threshold such as 10 gram), it means that there is noobject placed in this position, and this position is vacant.

Taking the neural network model as an example, a certain number oftraining samples may be acquired, and each position in each of thetraining samples may be labelled with first status information (whichindicates that there is an object in a position or there is no object ina position, for example). Then, the neural network model may be trainedusing the above training samples until a loss value determined from aloss function is less than or equal to a loss threshold. In this way,the current detection data may be input to the neural network model, andthe first status information of each position may be determined by theneural network model.

Taking the template matching model as an example, the statusdiscriminator may include a standard template that contains detectiondata collected when no object is placed in any position. Taking thedetection data including an image as an example, the statusdiscriminator may acquire a similarity between each position in thecurrent frame image and a corresponding position in the standardtemplate, which may be determined by using a cosine formula in therelated art. Then, the status discriminator may compare the determinedsimilarity with a preset similarity threshold (such as 0.9). When thesimilarity is greater than or equal to the similarity threshold, it maybe determined that the position in the current frame image is similar tothe corresponding position in the standard template, otherwise it may bedetermined that the position in the current frame image is not similarto the corresponding position in the standard template. That is, whenthe position in the current frame image is similar to the correspondingposition in the standard template, the processor 30 may determine thatthere is no object in the position; and when the position in the currentframe image is not similar to the corresponding position in the standardtemplate, the processor 30 may determine that there is an object in theposition, thereby obtaining the first status information.

It should be noted that when a camera or a pressure sensor is providedon the shelf, that is, when the current detection data includes imagedata or pressure data, the above-mentioned status discriminator may beconfigured to obtain the first status information. When a camera and apressure sensor are provided on the shelf at the same time, the pressuresensor may be configured for auxiliary detection. For example, it isdetermined that the first status information indicates the second statusthat there is no object in a position, which may include a situationthat a color of the object in the position is very similar to abackground color (for example, both are white), and the first statusinformation obtained in this situation may be inaccurate. For thisreason, the processor 30 may acquire pressure data of each position inthe second status. If the pressure data is less than or equal to apreset pressure threshold, it means that there is no object in theposition, and it may be determined that the second status is accurateand reliable. If the pressure data is greater than the preset pressurethreshold, it means that there is an object in the position, and it maybe determined that the second status is inaccurate. At this time, theprocessor 30 may adjust the first status information of the position tothe first status. In other words, the processor 30 may use the pressuredata to perform auxiliary verification on each position in the secondstatus, so as to solve the problem that the obtained first statusinformation is inaccurate when a color of an object in a position is thesame as or similar to a background color, which is beneficial to improvethe detection accuracy.

In practical applications, the processor 30 may perform the auxiliaryverification on a part of the positions that meet the requirements withthe pressure data. Taking the template matching model as an example, thestatus discriminator may calculate the similarity when determining thefirst status information. At this time, only when the similarity is lessthan the similarity threshold, and a difference between the similaritythreshold and the similarity is less than a difference threshold, theauxiliary verification is performed with the pressure data, therebyreducing the calculation. For example, the similarity threshold is 0.9and the difference threshold is 0.05, then only when the similarity isbetween 0.9 and 0.85, the auxiliary verification is performed with thepressure data.

At step 53, in response to the current detection data indicating that atarget position whose first status information indicates the secondstatus exists, first status information of the target position isacquired from previous detection data in a previous detection period.

In this example, the processor 30 may acquire a position whose firststatus information indicates the second status from the currentdetection data, which is hereinafter referred to as a target position.Then, the processor 30 may acquire the previous detection data in theprevious detection period of the current detection period, and acquirethe first status information of each target position from the previousdetection data.

At step 54, second status information of the target position isdetermined based on the first status information of the target positionin the current detection data and the previous detection datarespectively, where the second status information indicates a thirdstatus that an object is out of stock or a fourth status that an objectis picked up.

In this example, the processor 30 may determine the second statusinformation of each target position based on the first statusinformation of the target position in the current detection data and theprevious detection data, which may include steps 81-82 as shown in FIG.8 .

At step 81, the processor 30 may determine whether the first statusinformation of the target position in the previous detection dataindicates the second status, and the processor 30 may acquire a secondtime interval between time of the current detection period and time of alast detection period containing motion data, in response to the firststatus information of the target position in the previous detection dataindicating the second status. At step 82, the processor 30 may determinethat the second status information of the target position indicates thethird status in response to the second time interval exceeding a secondpreset time interval.

In an implementation, step 81 may further include determining that thesecond status information of the target position indicates the fourthstatus, in response to the first status information of the targetposition in the previous detection data indicating the first status.

In this example, the processor 30 may further control the display 40 todisplay the first status information or the second status information ofthe respective positions, as shown in FIG. 9 .

In this embodiment, current detection data of a shelf in a currentdetection period may be acquired; then when it is determined that thecurrent detection data satisfies a preset condition, first statusinformation of respective positions may be determined from the currentdetection data, where the first status information indicates a firststatus that there is an object in a position or a second status thatthere is no object in a position; after that, when the current detectiondata indicates that a target position whose first status informationindicates the second status exists, first status information of thetarget position may be acquired from previous detection data in aprevious detection period; and finally, second status information of thetarget position may be determined based on the first status informationof the target position in the current detection data and the previousdetection data respectively, where the second status informationindicates a third status that an object is out of stock or a fourthstatus that an object is picked up. In this way, in this embodiment, itmay be determined that the respective positions on the shelf includethree statuses: the first status, the third status, and the fourthstatus. That is, in this embodiment, it may be determined whether anobject in a position is out of stock or is picked up when the positionis vacant, thereby accurately reflecting a status of a position, whichis beneficial to improve an interactive experience.

A method of determining a status of a position on a shelf will bedescribed below in conjunction with a scenario in which a camera isprovided on the shelf, where the position is operatively configured toaccommodate and display an object. Referring to FIG. 10 , the method mayinclude a preparation phase and an operation phase.

I. Preparation Phase

1. A camera is mounted in front of the shelf and captures an imageinwards the shelf, and the processor may extract the image captured bythe camera in real time.

2. The image is divided into sub-images involving different object areasto be processed separately.

3. A standard template that is an image of an empty shelf is extracted,and background image data of each of the object areas is obtainedtherefrom.

4. A status discriminator is trained to determine whether each positionis vacant or occupied.

II. Operation Phase

1. The processor reads a frame of image from the camera as a currentframe image.

2. The processor performs motion detection on the current frame image.

3. When determining that the current frame image includes motion data,the processor updates “the last motion time”, ends the processing of thecurrent frame image, and returns to step 1 in the operation phase.

4. When determining that the current frame image does not include themotion data, the processor determines a first time interval, that is, atime interval between the current detection time and “the last motiontime”.

5. If the first time interval is less than or equal to a first presettime interval (such as 0.3 seconds), the processing of the current frameimage is ended, and step 1 in the operation phase is returned.

6. If the first time interval is greater than the first preset timeinterval, the processor determines first status information ofrespective positions in the current frame image, which indicates thereis an object placed in a position (a first status) or a position isvacant (a second status).

7. If it is determined in step 6 that the position is not vacant, theprocessor may update the status information of the position to the firststatus, end the processing of the current frame image, and return tostep 1 in the operation phase.

8. If it is determined in step 6 that the position is vacant, theprocessor may determine whether the position is vacant in previousstatus data.

9. If it is determined in step 8 that the position is not vacant in theprevious status data, “pickup time” is updated to the current detectiontime, it is determined that second status information is a fourth status(which indicates that an object is picked up), the processing of thecurrent frame image is ended, and step 1 in the operation phase isreturned.

10. If it is determined in step 8 that the position is vacant in theprevious status data, it is determined whether a time interval betweenthe current detection time and the “pickup time” (i.e., a second timeinterval) is long enough (for example, more than a second preset timeinterval such as 5 minutes). If so, it is determined that the secondstatus information indicates a third status (which indicates that anobject is out of stock).

It should be noted that the above-mentioned first preset time intervaland second preset time interval may be set according to variousscenarios. For example, for a scenario with a high timelinessrequirement, the second preset time interval may be set to be relativelyshort, such as 10 seconds, otherwise, the second preset time intervalmay be set to be relatively long, such as 5 minutes or even an hour.

On the basis of the above method of determining a status of a positionon a shelf, an embodiment of the present disclosure further provides anapparatus for determining a status of a position on a shelf, theposition being operatively configured to accommodate and display anobject. Referring to FIG. 11 , the apparatus includes a current dataacquiring module 111, a first status determining module 112, a firststatus acquiring module 113, and a second status determining module 114.

The current data acquiring module 111 is configured to acquire currentdetection data of the shelf in a current detection period, where thecurrent detection data includes data of one or more positions on theshelf.

The first status determining module 112 is configured to determine firststatus information of respective positions from the current detectiondata, in response to determining that the current detection datasatisfies a preset condition, where the first status informationindicates a first status that there is an object in a position or asecond status that there is no object in a position.

The first status acquiring module 113 is configured to: in response tothe current detection data indicating that a target position whose firststatus information indicates the second status exists, acquire firststatus information of the target position from previous detection datain a previous detection period.

The second status determining module 114 is configured to determinesecond status information of the target position based on the firststatus information of the target position in the current detection dataand the previous detection data respectively, where the second statusinformation indicates a third status that an object is out of stock or afourth status that an object is picked up.

In an embodiment, the first status determining module 112 includes:

a motion data determining unit, configured to determine whether thecurrent detection data includes motion data;

a first interval acquiring unit, configured to acquire a first timeinterval between time of the current detection period and time of a lastdetection period containing the motion data, in response to determiningthat the current detection data does not include the motion data; and

a condition satisfaction determining unit, configured to determine thatthe current detection data satisfies the preset condition when the firsttime interval exceeds a first preset time interval.

In an embodiment, the current detection data includes a current frameimage collected by a camera; and the motion data determining unitincludes:

a grayscale difference acquiring subunit, configured to acquire adifference in grayscale values between the current frame image and aprevious frame image at a same pixel;

a pixel number determining subunit, configured to determine a number ofpixels with the difference in grayscale values exceeding a presetgrayscale threshold; and

a motion data determining subunit, configured to determine that thecurrent frame image includes the motion data representing a motionrelative to the previous frame image, in response to a ratio of thenumber to a total number of pixels in the current frame image exceedinga preset ratio threshold, otherwise determine that the current frameimage does not include the motion data.

In an embodiment, the second status determining module includes:

a second interval acquiring unit, configured to acquire a second timeinterval between time of the current detection period and time of a lastdetection period containing motion data, in response to the first statusinformation of the target position in the previous detection dataindicating the second status; and

a third status determining unit, configured to determine that the secondstatus information of the target position indicates the third status inresponse to the second time interval exceeding a second preset timeinterval.

In an embodiment, the second status determining module includes:

a fourth status determining unit, configured to determine that thesecond status information of the target position indicates the fourthstatus, in response to the first status information of the targetposition in the previous detection data indicating the first status.

It should be noted that, the apparatus for determining a status of aposition on a shelf according to this embodiment corresponds to theabove method of determining a status of a position on a shelf. Fordetails, please refer to the contents of various method embodiments,which will not be repeated herein.

An embodiment of the present disclosure further provides anon-transitory computer-readable storage medium having a computerexecutable program stored therein, where the computer executableprogram, when executed by a processor, causes the processor to performthe above method of determining a status of a position on a shelf

In the present disclosure, terms “first” and “second” are only used fordescriptive purposes, and cannot be understood as indicating or implyingrelative importance. Term “a plurality of” refers to two or more, unlessspecifically defined otherwise. In the present disclosure, twocomponents connected by a dashed line are in an electrical connection orcontact relationship, and the dashed line is only used to make thedrawings clearer and the solution of the present disclosure easier tounderstand.

Those skilled in the art will easily conceive of other embodiments ofthe present disclosure after considering the specification andpracticing the disclosure disclosed herein. The present disclosure isintended to cover any variations, uses or adaptive changes of thepresent disclosure. These variations, uses or adaptive changes followthe general principles of the present disclosure and include commonknowledge or conventional technical means in the art that are notdisclosed by the present disclosure. The description and the embodimentsare to be regarded as exemplary only, and the true scope and spirit ofthe present disclosure are pointed out by the following claims.

It should be understood that the present disclosure is not limited tothe precise structure that has been described above and shown in thedrawings, and various modifications and changes may be made withoutdeparting from the scope thereof. The scope of the present disclosure isonly limited by the appended claims.

The invention claimed is:
 1. A method of determining a first status anda second status of a position on a shelf, the position being operativelyconfigured to accommodate and display an object, the method comprising:acquiring current detection data of the shelf in a current detectionperiod, wherein the current detection data comprises data of one or morepositions on the shelf; determining whether the current detection datacomprises motion data; in response to determining that the currentdetection data comprises the motion data, updating time of a lastdetection period containing the motion data, ending the currentdetection period, and returning to the step of acquiring the currentdetection data of the shelf in the current detection period; in responseto determining that the current detection data does not comprise themotion data, acquiring a first time interval between time of the currentdetection period and the time of the last detection period containingthe motion data; ending the current detection period, and returning tothe step of acquiring the current detection data of the shelf in thecurrent detection period, when the first time interval is less than orequal to a first preset time interval; determining first statusinformation of respective positions from the current detection data,when the first time interval exceeds the first preset time interval,wherein the first status information indicates the first status thatthere is an object in a position or the second status that there is noobject in a position; in response to the current detection dataindicating that a target position whose first status informationindicates the first status exists, updating a current status of thetarget position to the first status, ending the current detectionperiod, and returning to the step of acquiring the current detectiondata of the shelf in the current detection period; in response to thecurrent detection data indicating that a target position whose firststatus information indicates the second status exists, acquiring firststatus information of the target position from previous detection datain a previous detection period; in response to the first statusinformation of the target position from the previous detection dataindicating the first status, updating the time of the last detectionperiod containing the motion data to the time of the current detectionperiod, determining that second status information of the targetposition indicates a fourth status that an object is picked up, endingthe current detection period, and returning to the step of acquiring thecurrent detection data of the shelf in the current detection period; inresponse to the first status information of the target position from theprevious detection data indicating the second status, acquiring a secondtime interval between the time of the current detection period and thetime of the last detection period containing the motion data;determining that the second status information of the target positionindicates a third status that an object is out of stock, when the secondtime interval exceeds a second preset time interval; and controlling adisplay of the shelf to display the first status, the third status, orthe fourth status of the target position wherein the current detectiondata comprises a current frame image collected by a camera; anddetermining whether the current detection data comprises the motiondata, comprises: acquiring a difference in grayscale values between thecurrent frame image and a previous frame image at a same pixel;determining a number of pixels with the difference in grayscale valuesexceeding a preset grayscale threshold; and determining that the currentframe image comprises the motion data representing a motion relative tothe previous frame image, in response to a ratio of the number to atotal number of pixels in the current frame image exceeding a presetratio threshold, otherwise determining that the current frame image doesnot comprise the motion data.
 2. A non-transitory computer-readablestorage medium having a computer executable program stored therein,wherein the computer executable program, when executed by a processor,causes the processor to perform the method according to claim 1.